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Abstract —We consider a small cell network (SCN) consisting 
of N cells, with the small cell base stations (SCBSs) equipped 
with Nt > 1 antennas each, serving K single antenna user 
terminals (UTs) per cell. Under this set up, we address the 
following question: given certain time average quality of service 
(QoS) targets for the UTs, what is the minimum transmit power 
expenditure with which they can be met? Our motivation to 
consider time average QoS constraint comes from the fact that 
modern wireless applications such as file sharing, multi-media 
etc. allow some flexibility in terms of their delay tolerance. Time 
average QoS constraints can lead to greater transmit power 
savings as compared to instantaneous QoS constraints since it 
provides the flexibility to dynamically allocate resources over the 
fading channel states. We formulate the problem as a stochastic 
optimization problem whose solution is the design of the downlink 
beamforming vectors during each time slot. We solve this problem 
using the approach of Lyapunov optimization and characterize 
the performance of the proposed algorithm. With this algorithm 
as the reference, we present two main contributions that incor¬ 
porate practical design considerations in SCNs. First, we analyze 
the impact of delays Incurred in information exchange between 
the SCBSs. Second, we impose channel state Information (CSI) 
feedback constraints, and formulate a joint CSI feedback and 
beamforming strategy. In both cases, we provide performance 
bounds of the algorithm in terms of satisfying the QoS constraints 
and the time average power expenditure. Our simulation results 
show that solving the problem with time average QoS constraints 
provide greater savings in the transmit power as compared to 
the instantaneous QoS constraints. 

Index Terms —Small cell networks. Time average QoS con¬ 
straints, Virtual queue, Lyapunov optimization, CSI feedback. 


I. Introduction 

The global carbon emission related to information and 
communication technology (ICT) has been increasing at an 
alarming rate (an increase of 10% every year) 12 , 11 . There¬ 
fore, energy efficiency is becoming an important concern in 
the design of future wireless networks both from environmen¬ 
tal and economical point of view. Minimizing the transmit 
power leads to a significant reduction in the overall power 
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consumption at the base stations, hence leading to greater 
energy efficient design El. 

At the same time, there is an ever growing demand for 
higher data rate services and quality of service (QoS) guar¬ 
antees among mobile users. Network densification has been 
identified as a promising solution to satisfy the growing 
data rate demands, resulting in massive deployment of small 
cell (SCs) leading to greater spatial reuse a, 0. Never¬ 
theless, SCs alone cannot provide seamless coverage over 
large areas, and hence they must co-exist with the macro-base 
station, resulting in heterogeneous networks (HetNets). The 
deployment of SCs must be planned carefully (so that they 
can co-exist with macro-cellular networks), and hence low 
complexity, decentralized interference management schemes 
are very important in HetNet design 0, Q. 

Motivated by the aforementioned developments, in this 
work we consider the problem of minimizing the transmit 
power in SCNs, in which the small cell base stations (SCBSs) 
are equipped with multiple antennas. In HetNets, while the 
macro-base stations mainly provide coverage and signalling in¬ 
formation, the data intensive applications such as file sharing, 
multi-media etc. are transmitted by the SCBSs. Additionally, 
such applications also allow some latitude in terms of delay 
tolerance 0. Motivated by this fact, we consider the problem 
of minimizing the transmit power expenditure at the SCBSs 
subject to time average QoS constraints of the user terminals 
(UTs), where the QoS constraints have to be met over a 
long period of time (and not instantaneously). Time average 
QoS constraints provide the flexibility to dynamically allocate 
resources over the fading channel states as compared to 
instantaneous QoS constraints. In terms of energy savings, 
time average QoS constraint can lead to better performance, 
due to the fact that the transmissions can be delayed until 
favourable channel conditions are seen, thus minimizing the 
energy expenditure. The concept has also been exploited in 
the context of energy-delay trade offs 0, Qo). 

Related Work 

The issue of minimizing the downlink power and beamform¬ 
ing design in multi-antenna systems subject to UT signal to 
interference noise ratio (SINR) constraints was first addressed 
in im. The authors proposed an iterative algorithm based on 
uplink-downlink duality that converges to a feasible solution 
(if the solution exists). The aspect of feasibility of the down¬ 
link SINR targets, and the design of optimal beamforming 
vectors to minimize the transmit power was studied for a 
multi-user downlink scenario in ina and Ha. The downlink 
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power minimization problem was solved using a second order 
conic programming (SOCP) based approach in Oa. This 
result was interpreted in a Lagrangian duality based framework 
for the single cell and multi-cell scenario in na and m 
respectively. References ifTTll . ifTSll and lfT9ll proposed precoder 
design that maximize the so-called signal-to-leakage-and-noise 
ratio (SLNR) for all UTs simultaneously. However, all the 
aforementioned works consider the problem of beamforming 
design during a given time slot, for a fixed channel realization 
and instantaneous QoS constraints. 

The problem of handling delay optimal precoder design in 
multi-user MIMO systems has been addressed in l2Ql,|l2ll 
using the approach of Markov decision problem. However, 
these works are restricted to a single BS scenario, and do not 
address the issue of decentralized design, delayed information 
exchange, and CSI feedback constraints that are very essential 
in SCN design. 


Summary and Contributions 

In this work, we consider a stochastic version of the beam¬ 
forming design problem in SCNs, and consider minimizing the 
transmit power subject to the time average QoS constraints. We 
formulate our problem as a stochastic optimization problem, 
and propose a solution based on the technique of Lyapunov 
optimization ll22l . ||23]| . The Lyapunov optimization technique 
provides simple online solutions based only on the current 
knowledge of the system state (as opposed to traditional 
approaches such as dynamic programming which suffer from 
very high complexity and require a-priori knowledge of the 
statistics of all the random processes in the system). To the best 
of our knowledge, the application of Lyapunov optimization 
technique in the context of beamforming in MIMO systems 
under time average QoS constraints is novel. 

We model the time average QoS constraints as virtual 
queues, and transform the problem into a transmit power 
minimization problem under the queue stability constraint. We 
then use the technique of Lyapunov optimization to formulate 
the beamforming design during each time slot. We provide the 
performance bounds of this algorithm in terms of satisfying 
the QoS constraints and the time average transmit power. In 
our algorithm, the SCBSs can formulate the beamforming 
vectors using only the local channel state information (CSI). 
The SCBSs would only have to exchange virtual queue-length 
information among themselves. 

We then present two main contributions in the context 
of Lyapunov optimization that incorporate practical design 
considerations in SCNs. 

Delays incurred in information exchange between the 
SCBSs 

We introduce delays in information exchange between the 
SCBSs (for e.g., delays introduced in the backhaul links), 
and characterize the performance of our algorithm under this 
scenario. Specifically, we show that the delays in information 
exchange among the SCBSs result in only a constant gap 
with respect to the performance of the case with no delays. 
Moreover, under some conditions the gap vanishes, and the 
impact of delay on our algorithm becomes negligible. 


Limited CSI at the SCBSs 

Secondly, in order to limit the CSI feedback load, we consider 
the case when the SCBS can obtain the CSI from a limited 
number of UTs during each time slot. In this case, we solve 
the problem of joint CSI feedback and beamforming design 
by using the Lyapunov optimization framework. 

In practice, (e.g. in long term evolution (LTE) and LTE- 
advanced networks), obtaining the CSI feedback from all the 
UTs in the network becomes impractical as the number of UTs 
increase, since this leads to a huge feedback overhead (this 
is even true in a network consisting of single antenna links 
where the feedback is a simple scalar). Therefore, the SCBS 
have to decide which subset of UTs must feed back their CSI 
during each time slot. The problem of joint CSI feedback and 
transmission is known to be challenging. The main complexity 
lies in the fact that the transmitter must decide which UTs 
have to feedback their CSI without a-priori knowledge of their 
current channel states El. Eurthermore, in MIMO systems, 
CSI knowledge is crucial in formulating the beamforming 
vectors. The issue of joint CSI feedback and beamforming 
design problem (in terms of selection which UTs have to 
feedback their CSI) is very challenging. Most of existing 
works in this context either assume a predehned beamforming 
strategy (e.g. ZE), and/ or let all the UTs feedback a quantized 
version of their CSI El, ii, El, Ell. However, in practice, 
even when the CSI feedback is quantized, with a large number 
of UTs, only a subset of UTs can feed back their CSI (and 
choosing the optimal subset is complex). 

Eor the case of joint CSI feedback and beamforming design, 
we provide the following results; 

• We first present the feedback decision rule, and the 
corresponding beamforming design strategy obtained by 
the analysis of Lyapunov optimization. 

• Second, we present a low complexity algorithm named 
AlgF in order to optimally solve the CSI feedback 
decision problem obtained by the analysis of Lyapunov 
optimization. 

• We then provide a performance analysis of the proposed 
algorithm, and compare it to the performance of the 
optimal solution. We also show that under certain settings 
the performance can be made very close to the optimal 
value. 

The rest of the paper is organized as follows. We specify the 
system model in Section]^ We hrst provide the solution based 
on Lyapunov optimization with perfect CSI at the SCBSs 
in Section m The impact of delayed information exchange 
between the SCBSs is addressed in Section HVl The case with 
limited CSI at the SCBS and the problem formulation in case 
of joint CSI feedback and beamforming design is considered 
in Section|V] The numerical results are provided in Section VI 


Einally, the paper is concluded in Section VII The technical 
proofs of the results in this paper are provided in Appendices 
A,B, and C. 

Notations: Throughout this work, we use boldface lower¬ 
case and uppercase letters to designate column vectors and 
matrices, respectively. Eor a vector x, and x^ denote the 
transpose, the complex conjugate respectively (and similarly 
for a matrix). The notation tr(X), and A““(X) denote the 
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trace and the maximum eigenvalue of the matrix X respec¬ 
tively. We denote the identity matrix by the notation I. 

II. System Model 






2. 

{n,k) 


1,/c [f] 


where Xij[t] G C represents the information signal for the 
UTij during the time slot t and Zij[t] is the noise with 
variance Nq. The downlink SINR for UT^ ^ is given by 
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Fig. 1. Example of a SCN consisting of 2 small cells. 

The system model is illustrated in Figure [T] We consider 
SCN consisting of N cells and K user terminals (UTs) per 
cell The SCBSs are equipped with Nt antennas each, and 
the UTs have a single antenna. The notation UT^ ^ denotes 
the j-th UT present in the i-th SC. The SCBS of each 
cell serves only the UTs present in its cell. We consider a 
discrete-time block-fading channel model where the channel 
remains constant for a given coherence interval and then 
changes independently from one block to the other. We 
index the time slots by t. We denote the channel vector 
from the SCBSi to the UT^ ^ during the time slot t by 
G C^*. The elements of the channel vector are 
independent and identically distributed (i.i.d.). Furthermore, 
we consider that the channel vector can be represented by 
two components, i.e, where aij^k 

is the pathloss component and h' ^ [f] is the fast fading 
component, and E[h' ^ ^ = I. We define the 

channel matrix Hffl given Hi At] — [hi 1 [fl, ..., hi , [ill; 

H4fl = .„d H('] = : 

The channel process {H[f], < = 0,1,2,... } is assumed to be 
an i.i.d. (across time slots) discrete time stationary ergodic 
random process. We also consider a practical assumption that 
the channel gains are bounded at all times, i.e., 

Hynax yi,n,k. Throughout this work, we use the following 
definitions; 

• Local CSI at BSi ; The CSI from BS^ to all the UTs in 
the system, i.e., hi,n,k, n = 1,..., N, k = 1,... ^ K. 

• Non-local CSI at BSi '■ CSI from BSj (j ^ i) to 
UT„_fe, n=l,...,N, k = l,...,K. 

• Global CSI : CSI from all the BSs to all the UTs in the 
system. 

The local CSI corresponds to the information that can be 
obtained locally (either through feedback or uplink pilots in 
schemes such as the TDD), where as the non-local CSI is the 
information that must be exchanged between the BSs. 

Let us denote the beamforming vector corresponding to 
UTij during slot t by Wi j[f] G C’^*. The signal received 
by GTi j during time t is given by 


where the numerator represents the useful signal and the 
denominator terms represent the interference and the noise 
terms respectively. In the rest of this paper, we set Nq = 1, 
and normalize the useful signal and the interference signal 
power by the variance of the noise. Although not explicitly 
mentioned, henceforth all the signal powers are assumed to be 
the normalized values, and Nq is taken to be 1. The transmit 
power of SCBSi denoted as Pi[t] depends on the beamforming 
vector during the time slot t which can be given as Pi[t] = 
* = 1,..., The optimization problem 
to minimize the average energy expenditure subject to time 
average QoS constraint can be formulated as 


lim — 
T-foo T 


lim — 
T-foo T 


K 


i=i 


T-1 N 

^ — Q 2=:1 


(3) 

T-1 



E^bijW] > Vz,j 

(4) 




[tjWj j[t] E Ppeak 

yi,t, 

(5) 


+ Zij[t], (1) 

'in Subsection II.B, we provide extensions of the .system model to the case 
of a multi-tier network. 


where Ppeak is the peak power at which the SCBSs can 
transmit, jij is the QoS metric (to be specified) and Xij is 
the QoS target. We define QoS metric as follows: 

Ti.yW = \^Z[t]K^At]\^ 

H \'^n,k[*]^^,iAA^ + ^o) , (6) 

(n,fc) 

/Fj) 

where i/ij > 0 is a scaling factor. This QoS metric ensures 
that the time average useful signal power is greater than the 
time average interference signal power by a threshold Xij. 

The optimization problem described by Q is a 

stochastic optimization problem. The control action to be taken 
during each time slot is the formulation of the downlink 
beamforming vectors (Wi^j[f] Vijj) during every time slot t. 
Let Pinf be the minimum time average power incurred over all 
possible sequence of control actions that satisfy the constraints 
0-0. The problem in 0-0 can be solved optimally using 
techniques such as dynamic programming. But these methods 
are computationally complex and suffer from the curse of 
dimensionality. 

We propose to solve this by the method of Lyapunov 
optimization. Although this method is sub optimal, the control 
actions using this method can be computed with the knowledge 
of only the current state of the system and does not require 
a-priori knowledge of the statistics of the random processes 
associated with the system. Moreover, it has low computa¬ 
tional complexity as compared to dynamic programming based 
techniques. The application of this method transforms the 
stochastic optimization problem into a series of successive 
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instantaneous static optimization problems. Convexity of these 
instantaneous optimization problem is desirable to provide 
an efficient solution. However, the instantaneous optimization 
problems are not necessarily convex and depend on the form 
of time average QoS metric (as we shall see later in Section 

|m|. 

A. Comment on the QoS metric 

The QoS metric chosen in this work represents the differ¬ 
ence between time average useful signal power and the time 
average scaled interference signal power, which is constrained 
to be above a threshold value. One can view the QoS metric 
in similar spirit with metrics such as interference temperature 
control 1281, l29l . l30l, 1311 (in which the interference, peak 
interference power (PIP) or average interference power (AIP) 
is constrained to be below a certain threshold). Our QoS metric 
is more general in the sense that we constrain the interference 
signal to be below the useful signal power upto a threshold 
level. 

A QoS metric of more practical importance would be the 
time average SINK, and the associated constraint given by 

1 

lim -^E[SINR,.,[f]]> A,,,. (7) 

1 —^OO ± ‘ ^ 

However, the QoS metric chosen in (|^ (the difference form) 
has certain functional advantages in our algorithm design, 
which will be specihed in the rest of this subsection. Before 
enlisting them, we point out that with the help of numerical 
results, we will illustrate (in Section [Vl| i that the time average 
SINR constraint of 0 is indeed satished by the algorithm 
developed in this work. Therefore, it can be applied directly 
to problem with time average SINR constraint. 

The main drawback of using the time average SINR con¬ 
straint (in our algorithm design) is that the application of 
Lyapunov optimization technique with this metric leads to 
solving successive instantaneous optimization problems that 
are non-convex (as we shall see in the algorithm formulation 
in Section [nl| i. Consequently, we can only hnd a suboptimal 
solution to these non-convex problems. Furthermore, it is hard 
to characterize the gap between the solution provided by the 
Lyapunov optimization technique, and Pinf. It is also hard 
to hnd a decentralized implementation of this solution, and 
incorporate CSI feedback constraints. In order to overcome 
all the aforementioned issues, we introduce the modihed QoS 
metric in 0. The advantages of using this metric are the 
following: 

• It enables the development of low complexity beamform¬ 
ing solution to the stochastic optimization problem. It also 
enables us to analytically characterize the gap between 
the time average power expenditure of this algorithm and 
Pinf (recall that the solution obtained by the Lyapunov 
optimization technique is sub-optimal). 

• Furthermore, the beamforming design algorithm is de¬ 
centralized in which only the knowledge of local CSI is 
required, and the SCBSs only have to exchange scalar 
variables. 


• It enables the development of a low complexity joint 
CSI feedback and beamforming design. Furthermore, this 
algorithm is also decentralized. 


B. Extension to the case of Multi-tier Network 

The system model considered in this work can be extended 
to different Hetnet scenarios in a straightforward manner. To 
illustrate this point, note that instead of viewing the system 
as consisting of N small cells, one can simply consider the 
system as a network consisting of N tiers. The QoS constraints 
can now be stated as follows: 


n—1 

In (|^, |wf^ [t]hi i is the useful signal from the BS of the 
tier to UT^ in its own tier, and WP 

the interference arising from the BS of the n* tier (n f i) 
at the UTj in the z* tier, vf and ^ j can be interpreted 
as constants that scale the useful signal, and the sum of the 
interference caused by the BS of the n* tier on to a UTj in 
the z* tier. These constants can be set to suitable values in 
order to adapt the model for different Hetnet scenarios. The 
time average QoS constraint can now be set as 



4 K 

El 

\k=l 






MN-iVo. (8) 


1 

(9) 

1 — voo ± ' ^ 

Example: Consider a 2 tier network consisting of a macro¬ 
cell and a small-cell. Let us assume that the macro-cell is 
indexed by i = 1, and the small cell by z = 2. In this 
case, suppose the macro-cell users demand a certain average 
interference temperature constraint from the small cells. This 
can be accomplished by setting Vi=Q and vh = —1. The 
QoS constraint becomes 


T-l 


lim -VE 


T->c 


t=0 


K 




Lfc=l 


< Ai. 


( 10 ) 


Therefore, we can extend the system model of this paper to 
different HetNet scenarios. 

We now proceed to the algorithm design. In the rest of 
the paper, we use the short hand notation X to denote the 
time average value of the random process X\f\, i.e., X = 
limT^oo y 


III. Solution by Lyapunov Optimization Technique 

In order to model the time average QoS constraint, we use 
the concept of virtual queue The virtual queue associated 
with the time average constraint > A^ j evolves in the 
following manner, 

Qi,j[t + l] =max((5jj[f] -/Zij[f],0)-I-(11) 

where Alij[f] denotes the arrival process and tiij[t] denotes 
the departure process. The arrival and departure process are 
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given by 

(n,/c) 

/(*j) 

Note that the arrival and the departure process ca 
bounded as follows: 

^ -A^-^^max-Ppeak-^max “t” ^max-^0 “1“ ^i,j — ^ n 
^ -Ppeak-^max — /^max- 

The basic idea is to ensure stability of the virtual queue, stated 
mathematically as Ensuring the stability of 

the virtual queue implies that the time average of the arrival 
process is less than or equal to the service process, i.e. Aij — 
fLij < 0 Vi, j. In other words, the constraint Q is satisfied. 
Thus, we reformulate the original problem into minimizing the 
time average power expenditure while stabilizing the virtual 
queue as follows: 


N 


EA 

Z=1 


Adding the cost term (i.e. transmit po'^r during the current 

to both the sides of 


(12) 

time slot), VE[X)ij [f]w 

the equation (flSjl, we obtain 

(13) 

A(Q[f]) -f VE[^ w5[f]wi, 

Upper 

~ ^ ' E Qij [f] (Ai j [t] — p,ij 

max; 

(14) 

(15) 

Henceforth, we call the 

Q[f]] 


(16) 


VT 

( 20 ) 


The bound on the modihed Lyapunov drift for the queue- 
length evolution can be given as 

A(Q[f]) < Cl + 

i,3 

- E ® *3*J M (|w5 [tA 

'■A 

'id E - y^Z[t]^^At] QW] 

( 21 ) 


- I'i 


(n,fe) 


K 

S.t. Q-^3 < OO’ E < Ppeak- 

i,j j = l 


With this reformulation, we solve the optimization problem 
in 0 using the technique of Lyapunov optimization 
which allows us to consider the joint problem of stabilizing 
the queue and performance optimization. To this end, we 
define the quadratic Lyapunov function T* : M as: 

'h(Q[f]) = \'Aji,jiQid[AY■ The Lyapunov function is a 
scalar measure of the aggregate queue-lengths in the system. 
We define the one-step conditional Lyapunov drift as 


A(Q[f])=E vi/(Q[i + i]))_q/(Q[i]) Q[i] 


(17) 


where the expectation is with respect to the random channel 
states and the (possibly random) control actions made in 
reaction to these channel states. 

We now examine the Lyapunov drift corresponding to the 
evolution of the virtual queue Qij . 


Proposition 1. For the virtual queue which evolves according 
to ([n]), the Lyapunov drift follows the condition 


A(Q[t]) < Cl - Q^j[t]{Aij[t] - PiAA) QM 




Vf, 

(18) 


where 

Cl = NK{{A^^^f + (/r„,ax)') < oo, (19) 


where we have used the expressions for Ai^f] and p.ij\f\ 
from ( [T2 ] i and ( [T3] l respectively. 

Following the approach of Lyapunov optimization (drift- 
plus penalty method), we design the beamforming vectors 
during each time slot t to minimize the bound on the Lyapunov 
drift during each time slot ll22l . Before we proceed, we state 
the main intuition of using the Lyapunov optimization method 
in the design of the algorithm. 

The modified Lyapunov drift has two components, the 
Lyapunov drift term A(Q[f]), and the penalty term V x 
E[ [f]wi |Q[f]] term. Intuitively, minimizing the 

Lyapunov drift term alone pushes the queue-length of the 
virtual energy queue to a lower value. The second metric 

V X E [ ^j -wfj [f] Wi [f] IQ[t]] can be viewed as the penalty 
term for using high downlink power, with the parameter 

V representing the trade-off between minimizing the queue- 
length drift and minimizing the penalty function. Greater value 
of V represents greater priority to minimizing the downlink 
power at the expense of greater size of the virtual energy queue 
and vice versa. Therefore in our algorithm, we minimize the 
modified Lyapunov drift instead of only minimizing A(Q[f]), 
and obtain a trade-off between the two metrics of interest. 
Furthermore, we theoretically examine some properties of 
this algorithm and analyze its performance. In particular, 
we characterize the sub optimality in the performance of 
this algorithm with respect to Pinf, i.e. the minimum power 
expenditure over all feasible control policies 

Accordingly, the beamforming vector should be computed 
as a solution to the following optimization problem (minimize 


and Ajuax ond /imax represent the upper bound on the arrival 

and departure process specified in Gl and G3 respectively. ^The conditional expectation of the transmit power with respect to Q[f] 

is taken since the optimal transmit power during every time slot will be a 
The proof is provided in Appendix A, part 1. function of the virtual queue-length values. 
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the bound obtained in (|2T|); 


argmax^En Qij[i]|wf [i]|" 

(n,/c) 

#(*j) 


QM 


■ ( 22 ) 


where Eh indicates that the expectation is with respect to the 
random channel realization. 


A. Algorithm Design with Perfect Knowledge of Local CSI at 
the BS 


In this section, we assume that the SCBSs have the perfect 
knowledge of CSI of all its downlink channels (h^ „ ^ Vn, k) 
and examine the optimization problems ( |22l i. The CSI infor¬ 
mation can be obtained by feedback from the UT^ With 
the perfect knowledge of CSI, ( |2^ reduces to greedily mini¬ 
mizing the term inside the expectation (E[/(y)|y] = f{Y)). 
Therefore, we remove the expectation and solve the following 
optimization problem (we drop the time index t). 


max 


s.t. 


-VijQi.j 

(n,fc) 

^ ^ — -^eak Wi. 


We now examine ( |2T| in greater detail. The objective 
function of the optimization problem in ( |2^ can be rearranged 
and written as. 


Algorithm 1 (Decentralized Beamforming algorithm - DBF). 
During each time slot t, perform the following steps: 

• Compute j* = argmaxj 

• Set the beamforming vector corresponding to the UT j* 
as follows: 

(26) 

where )>o represents the indicator function 

whose value is 1 if > 0, and 0 otherwise, 

and XAmaxj-Ai jt) is the eigenvector corresponding to the 
maximum eigenvalue of matrix A,; . 

• For all other UTs j*), set 

w"f = 0 ill) 

From the steps of the DBF algorithm, it can be verihed 
that the optimal direction of transmission to solve ( |25l l is to 
transmit along the eigenvector corresponding to the maximum 
eigenvalue of the matrix Aij*, 

For simplicity of notations, we denote = 

and hence = tr(Aij[f]Wjj[t]). 

The solution implies that at most one UT can be active per 
cell during each time slot. Also, we can conclude that 

^tr(A,,,[f]W°f [f]) = PpeakA““(A,,,.)lA»^==(A,.,*)>0- 

3 

B. Properties of the DBF algorithm 

We now provide some properties of the DBF algorithm. • 
Intuition: Taking a closer look at the optimization problem 
( |25| ), it can be seen that each UT^ j has a metric associated 
with it given by. 


max 

(24) 





s.t. 

< Ppeak V* 

j 

^ ^ , fc Q' 



(n,fc) 

the matrix 




Useful signal 

Interference signal to other UTs 


-UW,.,). (28) 


{n,k) t3n^kQn,kYi-i^n,k YT and 

Aikj) _ _, 

Note that the optimization problem in ( |24| i is in separable 

form, in which each SCBS i can solve the sub problem given 
by 


max 

S.t. 


w^jAijWij 

3 

< Ppeak- 

j 


(25) 


We now provide an algorithm to solve the optimization prob¬ 
lem ( |25| ) and address it by the name Decentralized Beamfotm- 
ing algorithm - DBF. We also denote the beamforming vector 
corresponding to the DBF algorithm by Also, let us we 

denote X™^^{Aij) as the maximum eigenvalue of the matrix 


^A more practical case where the SCBS can obtain CSI feedback from only 
a limited number of UTs will be addressed in Section [V| 


The metric corresponds to the difference between weighted 
sum of the useful signal (to the UT^ j) and the weighted 
sum of interference caused to the other UTs in the system 
(UT(„, k)^(i,j))- The weights are the corresponding queue- 
length values which indicate how urgently the UT needs to 
be served. Therefore, intuitively, each SCBS schedules the 
UT in its cell which has the highest value of this metric 
). Additionally, the transmission direction corre¬ 
sponds to the eigenvector corresponding to the (A™ax(Aj )). 
The parameter V represents how aggressively the SCBS 
decides to transmit. Higher value of V implies less aggressive 
transmission and greater energy savings. 

The beamforming vector that maximizes the metric in ( |28| ) 
has some similarities with the “Leakage-based beamforming 
design’ ’im, ca, ESI, where in the beamforming vectors are 
chosen to maximize the SLNR metric. Since the original QoS 
metric considered in this work is in the difference form (as 
opposed to the ratio form of the SLNR metric), our solution 
chooses the beamforming vector that maximizes the weighted 
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difference between the useful signal and the interference 
caused to the other UTs in the system (and not the ratio as in 
the case of SLNR). Moreover, the useful signal power and the 
interference signal powers are scaled by the respective virtual 
queue lengths during that time slot. Therefore, our algorithm 
can be viewed as a “dynamic time varying leakage based 
algorithm” where the impacts of useful signal and interference 
signal are adapted dynamically according to the achieved QoS 
(represented by the virtual queue length levels). 

• Decentralized Solution: Observe that in order to formu¬ 
late the matrix Aij, the SCBSs only require the local CSl 
iKn,k Vn, k). The SCBSs would only have to exchange the 
queue-lengths (Qij) among themselves. Therefore, our formu¬ 
lation naturally leads to a decentralized solution. This results in 
tremendous reduction in the backhaul capacity requirements. 

C. Performance bounds for the DBF algorithm: 

The following proposition provides the performance bounds 
for the DBF algorithm; 

Proposition 2. The DBF algorithm yields the following per¬ 
formance bounds. The virtual queue is strongly stable and for 
any e > 0,C > 0, the time average queue-length satisfies 

^ 29 ) 

and the time average energy expenditure yields, 

(30) 

i 

Proof: The proof is provided in Appendix B, part II. ■ 

The bound of Proposition implies that the time average 
energy expended by the DBF algorithm can be made arbi¬ 
trarily close to the minimum average power (over all possible 
sequence on control actions) by increasing the value of V to an 
arbitrarily high value. This comes at the expense of increasing 
the average queue-length of the virtual queue. Intuitively, a 
high value of the average queue-length implies that the number 
of time slots required to satisfy the time average constraints 
is higher (analogous to the concept of delay in real queues). 

D. A Note on using the time average SINK for algorithm 
design 

Similar to the derivation at the beginning of ( |2T] i, it can 
be shown that the use of time average SINR (as the QoS 
metric) in the Lyapunov optimization method leads to solving 
the following instantaneous optimization problem during each 
time slot t : 


max 

[QljWsinr,., - 


(31) 

s.t. 

< Ppeak 

Vz. 
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Notice that the optimization problem in corresponds 
to solving a maximization of the weighted sum of SINR 
terms, which is a non-convex problem, and finding the global 
optimum is a non-trivial task. Additionally, it is very difficult 


to obtain a decentralized formulation of the solution corre¬ 
sponding to (0, and to develop an efficient CSI feedback 
strategy in the case when there are CSI feedback constraints. 
This highlights the functional importance of using the QoS 
metric in (|^ in our algorithm design. 

Next, we introduce delays in the information exchange 
among the SCBSs. 

IV. Delayed Queue-length Information Exchange 

In this section, we make a novel contribution by studying 
the impact of delayed information exchange between SCBSs 
(such as the delays introduced in the backhaul). Recall that 
in our beamforming solution, the SCBSs can formulate the 
beamforming vectors using only the local CSI. However, the 
SCBSs have to exchange the queue length information (scalar 
values). In this section, we show that this exchange does not 
have to be in real time, and our solution can be made arbitrary 
close to infimum even if the exhcange is done with delay. We 
study the proposed solution in presence of delayed information 
exchange between SCBSs, and show the impact of this delay 
on the gap between our solution, and the minimum transmit 
power of the original stochastic problem. 

A. Algorithm Design with Delayed Information Exchange 
between the BSs 

Let us assume that a delay of r < oo time slots is incurred 
while the SCBSs exchange the queue-length information. Each 
SCBS i now has perfect queue-length information of its local 
queues (Qi,j[t] Vj) and the delayed queue-length information 
from the neighboring queues iQn,k[t — t], Vn f i,k). 
Note that our set up can be easily generalized to introduce 
different delays Tnfinfi corresponding to the queue-length 
information from different SCBSs. However, in order to keep 
the notations simple, we restrict ourselves to uniform delays 
(r, Vn ^ z). We assume that the SCBSs treat the delayed 
queue-length as the true value of the queue-length. Every 
SCBS now solves the following optimization problem, 

max ^tr(AA[f]W,j) (32) 

3 

S.t. ^tr(Wij) < Ppeak 
3 

where the matrix [t] is given by, 

- - LI. (33) 

n^i,k 

Let us denote the solution corresponding to optimization prob¬ 
lem ( |3^ by We will henceforth use the superscript 

’’del” to denote parameters corresponding to the solution of 
(|32 1 . Once again, following similar argument as the solution 
to (2^, it can be shown that at most one UT can be active 
per cell. 

Algorithm 2 (DBE algorithm with delayed information ex¬ 
change). During each time slot t, perform the following steps: 







• Compute j* = argmaXj 

• Set the beamforming vector corresponding to the UT j* 
as follows: 

(34) 

where lA"'“(Ai j»)>o represents the indicator function 
whose value is 1 if > 0, and 0 otherwise, 

avid X^max^^. ^ is the eigenvector corresponding to the 
maximum eigenvalue of matrix A.ij*. 

• For all other UTs jff j*), set 

xvt:‘ = 0 (35) 


It is clear that the optimal power allocation policy is given 
by 


del ^ j ^peak if j = J>nd ) > 0 

|0 else, 
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and the time average queue-length satisfies 

^< C^+C, + VNKP,,,, 

where Ci is defined in < P^ and C 2 defined from Lemma^ 

Corollary is proved in Appendix B, part II. 

The corollary shows that by increasing the value of V, the 
time average power expenditure can be made very close to 
the optimal value of the original problem, i.e., the impact of 
the delays in information exchange can be made negligible. 
However, this will have an impact on the time needed to 
achieve the time average QoS constraint (i.e., we need greater 
number of time slots to achieve the QoS). This result shows the 
SCBSs do not have to exchange the queue-length information 
in real time. The SCBSs can delay this exchange, and even 
adapt it to the backhaul capacity/load. Under some cases, the 
impact of the delay in information exchange can be even made 
negligible (for e.g., choosing a high value of the parameter V). 


and therefore, 

max^tr(A[_^ [f]W“[f]) = PpeakA““(A,,j*)lAm.x(A, ^.)>o- 

'(37) 

We ow theoretically examine the performance of the DBF 
algorithm with delayed queue-length information. 

B. Performance Analysis with Delayed Information Exchange 
Between the BSs 

We first compare the performance of DBF algorithm with 
respect to the solution of ( |3^ (i.e. the case with delayed queue- 
length exchange) in the following lemma. 

Lemma 1. There exists a constant 0 < C 2 < oo independent 
of the current queue-length Qij[t], yi,j such that, 

Y,triA,At]W?f[t]) <J2tr{A,Amtj[t]) + C2 W. 

(38) 

Proof: The lemma is proved in Appendix B, part I. ■ 
Lemma [T] states that the performance of the DBF algorithm 
with delayed queue-length information exchange differs from 
that of DBF algorithm (with instantaneous queue-length in¬ 
formation exchange) by a bounded constant. The key element 
in this lemma is the fact that the constant C 2 is independent 
of the current queue-lengths which will be helpful in proving 
the performance bounds for the DBF algorithm with delayed 
queue-length information exchange. 

Corollary 3. For the DBF algorithm with delayed queue- 
length information exchange, the following performance can 
be obtained for any e > 0 and V > 0.' The time average 
transmit power expenditure satisfies 

( 39 ) 

i=l 


V. Joint optimization of the CSI Feedback and 
Transmission 

In this Section, we solve the problem of joint CSI feedback 
and beamforming design by using the Lyapunov optimization 
framework. The solution itself is not straightforward from 
the Lyapunov optimization technique, and requires several 
intermediate proofs that are presented in Theorem]^ 

A. Algorithm Design : Joint CSI Feedback and Transmission 

CSI Feedback Model ; In practice, the UTs feedback 
a quantized version of the CSI to the SCBS. However, in 
this work, owing the complexity of the joint feedback and 
beamforming design problem, we consider a simple feedback 
scheme. Under this scheme, we assume that if SCBS^ decides 
to obtain the CSI feedback from the UT„ fe, then the UT can 
feedback this information perfectly. Hence, SCBS) has the 
knowledge of the exact value of „ fc. For the rest of the UTs 
(that do not feedback their CSI), SCBSi assumes the channel 
to be the mean value, i.e. E[hi „ j,]- Consequently, the SCBS 
must decide which of the UTs have to feedback their CS0 We 
impose the following feedback constraint: during every time 
slot t, the SCBS can obtain feedback from at-most Umax UTs 
(recall that in practice, the SCBS can obtain the feedback from 
only a subset of the UTs). We denote the indicator variable 
for the CSI feedback decision by 

J1 if UT„ k feeds back its CSI to SCBS^ 

" jo else. 

The feedback constraint can then be stated as 

^ ^ ^i,n,k ^ -^max- (41) 

n,k 

‘^The impact of channel quantization errors will be addressed in the future 
work 
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Let us denote hij = [bij^i,... ,bij^K]', and = 

[bi 1 ,..., bi jv] and b = [bi,..., bAr]. Also, for simplicity 
of notations, we henceforth denote 


If f is selected, then update Qu as Qu = QuXQij' or if 
k' is selected, then update Qii_ as Qii_ = Qu- \ Qn k'- 
IfEn.kb i,n,k If f?max, go to Step 2, else terminate. 


Qi,n,k — Qi,n,k^i,n,k' 

The joint CSl feedback decision and transmission algorithm 
can be derived by following steps similar to Proposition [T] 
and the beamforming vector design of ( |2^ (omitted here 
for brevity). In what follows, we first present the joint CSI 
feedback decision and transmission rule obtained from the 
Lyapunov optimization technique (for the case with partial 
CSI), and then present the results on the performance analysis. 


Algorithm 3 (Joint CSI Feedback and Transmission Algo¬ 
rithm). During each time slot t, perform the following steps: 
• Choose the CSI feedback decision as a solution to the 
following optimization problem: 


b™ = 


maxE 

b 


max 

p 




,j 




^ ^ k^n,kQn,k(bi,n,k^^i,n,k “f ’ 

{n,k) 

(42) 


where we denote b^^ to be the resulting CSI feedback 
decision. 

• After computing b^^, if ^ = 1, set the channel 
vector from SCBSi to UTn^k to (i.e., the actual 

value of the channel realization). Else, set the channel 
vector from SCBSi to UTn,k to After setting the 

corresponding channel values, compute the beamforming 
vectors by repeating the steps of Algorithm 1. 


A Low Complexity Algorithm to Solve 

Notice that the feedback design problem in ( |42] l is a 
stochastic optimization problem with discrete variables b, 
and over the power allocation P. We propose the following 
low complexity implementation scheme, labeled AlgF to 
optimally solve ( |4^ . Before we do so, let us define 

Qu = {Qi,i,Qi, 2 , ■ ■ ■ ,Qi,K} (43) 

Qii- — k^n,2Qn,2^ • ■ • : k^n,KQn,K} '^Tl %. (44) 

AlgF can be implemented in the following manner: 


Algorithm 4 (AlgF). Perform the following steps. 

• Initialize bi^n,k = 0, Vn, k. 

• Choose the index j' = argmax^ Q^^ and k' = 

arg maxj, „^, Qu_ . Then, select either the index f or k' 
that maximizes the cost function in If f is selected, 

then set bi^iji = 1 or ifk' is selected, then set bi^n.k' = 1- 


The aforementioned algorithm requires the computation of 
the max eigenvalues for UTs. This results in a huge 

complexity reduction as compared to the exhaustive search 
method that requires the computation of the max eigenvalues 
for combinations. 

It is worth noting that, in practice, the UTs in other cells 
i.e. (n f i, k) may not be allowed to feedback their CSI 
to base station i. Under this condition, a similar feedback 
problem can be reformulated. The constraint ( |4T] ) is now over 
the UTs of cell i. Our feedback algorithm can be modified 
taking this constraint into account, which results in a simple 
algorithm that consists of selecting the i?niax UTs that have 
the highest Qtj (the computation of the maximum eigenvalue 
is not needed in this case). 


B. Algorithm Performance Analysis 

In this subsection, we provide the performance analysis of 
both AlgF and the joint CSI feedback decision and transmis¬ 
sion algorithm. 


Theorem 4. The proposed feedback algorithm AlgF is the 
optimal solution to the problem in 


The proof is given in appendix C, Part I. 

Let P/jjf be the minimum time average transmit power 
P, over all possible sequences of control actions of the 
optimization problem 0, with an additional constraint on the 
CSI feedback as in (|4^. 


Corollary 5. The proposed joint feedback and beamforming 
algorithm can achieve the following performance : The aver¬ 
age energy energy expenditure, and the time average virtual 
queue length satisfy the bounds given by 




Cl 

V 


^ Cl -|- VNKPpeak 

2_^ Lii j S ————— 
hi 


(45) 


The proof is provided in Appendix C, Part IT Recall that 
the time average QoS constraint is satisfied if the time average 
queue lengths are bounded, which is ensured by the use our 
algorithm for finite values of V, N and K. 

VI. Numerical Results 

In this section, we present some numerical results to demon¬ 
strate the performance of the DBF algorithm. We consider a 
system consisting of 2 small cells with each cell having 2 UTs 
each. Each SCBS has 5 antennas and Ppeak = lOdB per SCBS. 
We consider a distance dependent path loss model, the path 
loss factor from SCBSi to UTj^fc is given as crij^k = 
where U the distance between SCBSi to UT^ fc, nor¬ 

malized to the maximum distance within a cell, and /? is 
the path loss exponent (in the range from 2 to 5 dependent 
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on the radio environment). Once again, as stated before, we 
assume Nq = 1, and normalize the signal powers by the noise 
variance. 

We plot the time average energy expenditure per SCBS 
versus the target QoS for two cases. In the first case, we 
solve the problem of minimizing the instantaneous energy 
expenditure subject to instantaneous QoS target constraints 
( s.t. VQ. We 

repeat this for 1000 time slots. In the second scenario, we solve 
the problem of minimizing the time average energy expendi¬ 
ture subject to time average QoS constraints (jij > Xij). We 
plot the result in Figure It can be seen that for the case with 
time average constraints, the energy expenditure is lower. In 
particular, for a target QoS of lOdB, energy minimization with 
time average QoS constraints under the Lyapunov optimization 
based approach provides upto 4dB reduction in the energy 
expenditure as compared to the case with instantaneous QoS 
constraints (for V = 800.) This is in accordance with our 
intuition that the time average QoS constraint provides greater 
flexibility in allocating resources over channel fading states. 


0) 

o 

(1. 

.3 

o 

Q 



Fig. 2. Average energy expenditure Vs target QoS for a two cell scenario, 
each cell consisting of two UTs, Nt = 5, Ppeak = lOdB. 


0) 

tiO 

a 

> 


0) 

> 



Fig. 3. Achieved time average SINR Vs target QoS for a two cell scenario, 
each cell consisting of two UTs, Nt = 5, Ppeak = lOdB. 


frequently resulting in higher average queue-length and lower 
average energy expenditure. 



We also plot the achieved time average SINR as a function 
of the target QoS for different values of V in Figure 0 It 
can be seen that in each of the cases, the achieved time 
average SINR in the downlink is above the target value 
Xij. This result emphasizes the fact that although the QoS 
constraint of the form f© was used in the algorithm design, 
the resulting algorithm still achieves the target value in terms 
of the achieved time average SINR. Thus, our algorithm can be 
directly used in the case with time average SINR constraints. 

We next plot the time average energy expenditure per SCBS 
versus the average queue-length for different values of V 
obtained by running the DBF algorithm for 1000 time slots 
in Figure The target time average QoS is lOdB. It can be 
seen that as the value of V increases, the time average energy 
expenditure decreases, and the average queue-length increases. 
This is in accordance with the performance bounds of DBF 
algorithm. Increasing V implies that the SCBS transmits less 


Fig. 4. Time average queue-length vs time average energy expenditure for a 
two cell scenario, each cell consisting of two UTs, Nt = 5, peak power per 
SCBS = lOdB Target QoS value= lOdB. 

Next, we examine the impact of the number of transmit 
antennas on the target QoS. We plot the average queue-length 
(of the virtual queue) as a function of the target QoS for 
different number of transmit antennas in Figure First, it 
can be seen that as the number of transmit antennas increase, 
the average queue-length becomes lower. Also, it can be seen 
that there is a cut-off point beyond which the average queue- 
length blows up. The cut off point represents the maximum 
supportable QoS target for the system. The higher the number 
of transmit antennas, higher is the maximum supportable QoS. 
This is due to the fact that higher number of antennas leads 
to greater degrees of freedom resulting in enhancement of the 
useful signal power and less interference power. 
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Fig. 5. Time average queue-length vs target QoS in dB for different number 
of transmit antennas, peak power per SCBS = lOdB, V = 100. 


Appendix A; Performance Bounds 
Part 1: Proof of Proposition^ 

From GB, we can write the following. 

+1] - (QijM “ W 

+ 2Aij [f] max (0, Q*^ [t] - pij [t]) 

^ Qij [^] + Ai,j W + [^] 

- 2Qij [f] {pij [t] - Aij [f]). (46) 

Summing with respect to i,j and taking the conditional 
expectation E[.|Q[f]], we have, 

Am]) <Y,E[pi[t]+Aiimit]] 

'id 

- X] [Mi.i W “ WIQW] (47) 

id 

Using the bound Pij[t] < ^imax and Aij[t] < A^ax, and 
defining Ci = Ei,i((^mix)^ + (plkiAA- Taking the upper 
bound AJ^^ax < ^max and < Mmax, we obtain Ci = 

-^7tr(A^ax + Fmax)) we obtain the bound of ( fTS] ). 


VII. Conclusion 


A. Part II: Proof of Proposition 

From (|2T]), for the DBF policy we have, 


In this work, we handled the problem of minimizing the 
transmit power expenditure subject to satisfying time average 
QoS constraints in a SCN scenario. Using the technique of 
Lyapunov optimization, we proposed a decentralized online 
beamforming design algorithm whose performance in terms 
of the time average power expenditure can be made arbitrarily 
close to the optimal. Our results show that time average QoS 
constraints can lead to better savings in terms of transmit 
power as compared to solving the problem with instanta¬ 
neous constraints. Additionally, we showed with the help of 
numerical results that the achieved time average SINR with 
our algorithm also satisfies the target value. In addition we 
also considered two practical cases of interest in SCNs. In 
the first, we considered the impact of delay in information 
exchange among the SCBSs. We showed that the performance 
of the proposed algorithm with delays is only affected upto 
a finite constant in comparison with the case of no delays. 
Secondly, we considered the impact of CSI feedback. We 
formulated a joint CSI feedback and beamforming frame¬ 
work using Lyapunov optimization technique. Furthermore, 
we provided a low complexity algorithm to optimally solve 
the CSI feedback problem (that is obtained by the analysis 
of Lyapunov optimization). We then provided performance 
bounds between our solution and the optimal solution of the 
original joint feedback and beamforming design stochastic 
problem. 


A(Q[i]) + UE[^(w°f)^[t]w°f [t]|Q[t]] < Cl 
id 

id 

id 

(n,/c) 

-U(w°f[t])«w°f[t]|Q[i]] (48) 

(a) 

— T / ^ Qid M i^id T 

id 

id 

(n,fc) 

- V (w5 [i][t] IQ[i]] (49) 

where the beamforming vector wj® is the one implemented 
with any stationary randomized policy. Inequality (a) is true 
due to the following reason. Recall that the DBF algorithm 
is implemented to maximize the RHS of the bound in ( |48] l. 
Therefore, replacing ( |48l l with any other control policy should 
yield the inequality of (a). 

In particular we replace it by a stationary randomized policy 
which satisfies the following conditions. 
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^ |(w’^®n,fc[t])^h„,i,j[t]|^ - t'ijAro|Q(t)] >Aij+e, 

{n,k) 

(50) 

i,3 

The existence of such a policy is guaranteed by arguments 
from Caratheodory’s theorem and its proof is omitted for 
brevity (the reader can refer to ll22l for the details of the proof). 
Using ( |50| and (|5T| in ( |49] l yields, 


where = Qn,k[^ - - Qn,k[t]- Using ([S^ in the 

inequality ( |53| l yields, 

5]tr(A[,^.[f]W°f[f]) 

3 

j n^i^k 

3 

+ E E ^n.feQlfcWtr(H,,„,fc[f]W"[f]) (56) 

j n^i,k 

Adding ( |54l i to ( |5^ yields. 


A(Q(f)) + UE[^(w°f [f])«w°f [f]|Q[f]] 

< Cl + E/ ~ E/ * 5 *J + ^) + ^^inf 

= NKBi - e Y, Q^,j + yPM (52) 

Using the result of ( |5^ , and following some straightforward 
steps (similar to Lemma 4.1, m, omitted here for brevity) it 
can be shown that. 


T-l 




Cl + VNKP^^ 


i,j 

T-l 




i 


V 


E E ^n,fcQEMtr(H,,„.4f](W°f[f]-W“[f]))>0. 

j n^ijk 

(57) 

Finally consider the term tr(Aij[f](W™^[f] — 

Using ( |55l l, we obtain 

^tr(A,,[f](W°f[t]-W“[f])) 

3 

= ^tr(A[,^.[f](W5^f[f]-W“[f])) 

3 

j n^i,k 

<E E ^n,.QlfcMtr(H,,„,fc[f](W™P[f]-W“[f])) 

j n^i,k 

(58) 


Appendix B ; Performance with Delayed 
Queue-length exchange 

B. Part I: Proof of Lemma 

From the definitions of and we can 

conclude the following 

^tr(A,.,[f]W°f [f]) > ^tr(A,,,[t]W“[f]) (53) 

3 3 

^tr(A[,^.[f]W“[f]) > ^tr(A[,^.[f]W°f [f]) (54) 

3 3 

Recall the expression for given by 

J [^] ^ ^ ,fcQz,fc [^]^i^fc [t] 

^ ^ ^n,kQTi,k[i]'^i,n,k[i] ^ 
n^i,k 

Adding and subtracting J2n^i,k’^3i,kQn,k[t - on 

the right hand side, we obtain 

[^] — J [^] ^ ^ [t] 

^ ^ k'n,kQn,k[i 

n^i,k 

E ^ ^ k'n,k{Qn,k[i '^] Qn,/c [^] [t] V 

n^i,k 

= Al^ [t] + Y ^n,kQik m^,n,k [t] (55) 

n^i,k 


where the last inequality follows due to the fact that (from 

(lU) 

y]tr(A[,^.[f](W°f[f]-W"[f]))<0. 

3 

Using the equation for queue-length evolution in ( [TT| l and 
the bounds in Aij[t] < Amax and < /imax, we can 

conclude the following. 

Qtj [t-'>'] + max >Qt,j[t] V*,j 

t] Tpaiax ^ [^] '^fj (59) 

Combining the above equations yield. 


Qi,j W “ max E ^ QiAA + ''"M max; yi,j 

(60) 


And hence we can conclude that 


TAaaax ^ — T/Xniax '^fj ( 61 ) 

Note that in ( |57l i, we have already shown that the right hand 
side of ( [58] l is positive. 

In order to bound the right hand side of ( |58| ), we proceed as 
follows. First, we recall that there can only be one active UT 
per cell. Therefore, only one of is a non-zero matrix. 

Similarly, the case for Therefore, 

j n^i,k 

E 2 maxlTTlpQax)'^/^max}-Ppeak rUUX |[^] | — (62) 
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Therefore, 

^ti-(A,,,[f]W°f [t]) < ^tr(A,,,[f]W“[f]) + B, Vz,f. 

j 3 

and hence, 

^tr(A,., [f]W°f [f]) < 5]tr(A,,,[f]W“[f]) + 

(63) 

where C 2 = NBi < 00 . 

Part II : Proof of Theorem 
Rewriting ( |2T| i, we have 

A(Q[t]) + yE [ ^ [t]w,j [f] IQ[f]] < 

= Cl + Q^,J W (Kj + - Y. MW,.,M) 

(64) 

where we have used the equivalence of the quadratic form 
and the trace form. With delayed queue-length information, 
the policy corresponding to W‘^®* is implemented. Therefore, 
we have 

A(Q(t)) yE[^(w“[i])^w“[t]|Q[t]] 

< Cl+Y - 5] tr(Ai.,[t]W"[i]) 

(a) 

< C\ + 'y ^ Qi,j[t]{^i,j + Vi^jNf) 

i,3 

- (tr(A,,,[i]W°f [i]) - C 2 ) 

= Ci+C 2 + Y QiAAiKi + - tr(Ai,, [t]W°f [i]) 

(65) 

where (a) follows from Lemma [T] Now following similar 
steps as Appendix A, part 11, we can obtain the bounds of 
(|4^ and 

Appendix C; CSI Feedback Scheme 
Part I: Proof of Theorem 

Let be the solution obtained by our feedback algo¬ 
rithm. First note that Yl,n,k K,n,k = Let S* be the set 

of UTs that feedback their CSI according to our algorithm to 
the SCBS, i.e. = 1 V(n, k)_ G S'*. Let S* the set of UTs 
such that b*^ f. = 0 y{n,k) G S*. We prove the theorem by 
showing that if we replace any UT in S* by a UT in S*, the 
resulting objective function of will have a smaller value. 

We examine the following 3 cases; i) both UTs are in the 
cell i, ii) both UTs belong to other cell {n f=- i) and iii) one UT 
belongs to cell i and the second belongs to other cell (n i). 
Lets use the notation Hi n k = o'i n kH'i n k- '^6at 

Kn,k and nr are independent of each other for k k', 
and = E[H'.„.,] = I. 

i) First, lets consider two UTs j and f in cell i such that 
= 1 and 6*,.,, = 0. Clearly Q,., > (by the 


steps followed in our algorithm). Lets consider the objective 
function of @i.e. 

E[max5] -I- 

^ ^ ^n,kQn^kipi^n,k^i,n,k 4" Hz,n,fc) (6^) 

(n,/c) 

and denote by matrix = 

/c) where 

is the matrix that contains the channels between the 
base station i and all UTs in other cells n i. In order to 
simplify the proof description, lets consider that there exists a 
third UT k in cell i that lies in set S* (however our argument 
holds for any number of UTs in the system). One can notice 
that for every channel realization, the solution corresponding 
to the optimal beamforming and power allocation in 
is such that only one UT is active, or none of the UTs are 
active. Therefore, Pij is either Ppeak or 0. According to our 
feedback algorithm 6*^, = 1 and the objective function in 
( |42| corresponding to UT j is 

k'i j/Qi j'H 

Fij > 0 and Fij > Fi^k for some channel and virtual queue 
states i.e. setting 6* j, = 1 will increase the objective function 
of ( |42l i. If we interchange UTs j and f i.e. we set 6*^, to 0 
and bP J, to 1, the objective function in ( |42] i corresponding 
to UT j' is 


Ui jQi jH 

— PiJ'VijQiJ 

(b) . (c) 

< P,.,-[A“""(g,./(H'.,.,0) - < 0. (67) 


Therefore P^,/ is set to 0 (in order to maximize the objective 
function of ( |42| l) and P^.,/ = 0 for all channel states. Notice 
that (a) follows from H'i ^, = I and (b) follows from 

i,j' > 0 V Wi.,/. (c) follows 
from Qij > Qi,j', A™“(HU ,-,) = 1 and > 1 (target 
SNR is higher than 0 dB). Consequently, setting b*j, to 0 
and to 1 will reduce the objective function of ( |4^ . 

ii) Lets now consider two UTs not belong to a cell i, denoted 
by the index (n, k) and (n', k'). UT (n, k) G S* i.e. = 1 
and {n',k') G S* i.e. = 0. Therefore i^n',k'Qn',k' < 

k'n,kQn,k- The objective function of any UT j belonging to 
cell i is 


= P,.,A““(Q,,,(HF.,) 

^n,kQn,k^^i^n,k ^n^k'Qn^k'^^ 

= P,.,[A“^^(Q,.,(H'.,.,) - C,.,(H„_) - o^,kQu.kKn,k) 

^n' ,k'Qn' ,k'] ( 68 ) 
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where matrix = Y.(i,k)’^i,kQiAbi,i,k'iii,i,k + 

iii.i,k) + where the sum is over all UTs in all cells {I, k) 
(including cell i) except UTs {n,k) and (n',k'). llii_ 

is then the matrix that contains the channels between the base 
station i and all UTs in other cells except channels 
'tli,n,k and Hi „,/fe/. Notice that (a) follows from H'i „/fe/ = I. 
The objective function of (|42li can be written as follows 


E- 


Hi 


,H 


E- 


h; 


max 
- p 




I Hi, ,H' 






If we change the feedback decision by setting b* ^ ^ = 0 and 
b* y = 1, the objective function of UT j becomes, 

- Ci.,(Hi,_) 

^n' ,k'Qn' ^n,fcQn,fc] 

The objective function of ( |42] t becomes, 

[maxy^U^j |Hij_,H-_ij] (69) 


Let Fij(H) and F/^ jH) be the objective function of UT 
ii,j) when respectively H'„ = H and H'= H. By 

definition of A™“, F/j (H) is given as, 

FF (H) = Pi,, [ max wf, (H',,,,) - Q,, (Hi,_) 

k'n\k'Qn\k'^)^iJ k^n,kQn,k 


= P; 




J*H 


,,(Qi,,(H',i,,)-Ci,,(Hi,_))w'L 


- Pn',fc'Qn',fc'W'*'^i,,(H)w'L - iyn,kQn,k] 


^=^P: 


w-5(Qi,,(HU,,)-Ci.,(Hi,_))w- 

k'n.kQn.k^ t,j(H)Wi,,- k'n' ,k'Qn' ,k' 

+ (1 - w'*-^i,,(H)w-L)(p„/,fe/(5„/,fe/ - Vn,kQn,k) 


{b) 

< P^,3 


w'*5(Qi.,(H',i,,)-Ci,,(Hi,_))w'L 


k'n,kQn,k^ 


— Pi,j 


max Wi,, (Qi,, (HU,,) - C'i., (Hi,_) 

llwi.ill =1 

(c) 


Pn,/cl5n,fcH)wi,,- k^n' ,k'Qn' ,k' — J (^) (^0) 


where (a) follows by adding and subtracting Vn' ,k'Qn' ,k' and 
Pn,fcQn,fcW'*^i,,(H)w'*^. (b) follows from Vn',k'Qn',k' < 
Vn,kQn,k (due to OUT feedback algorithm since (n, k) G S* 
and {n'\ k') G S*) and (H)w'^ < 1 ( A““(H'_„_fc = 

H) = 1 since Hi,„,fc = cri,„,fcH'„where cri,„,fc is the 
average channel gain), (c) follows from the definition of 
Therefore, for a given channel state H and for any UT {i,j) 
we obtain P/, (H) < Pi,, (H). Using the fact that H' and 


are i.i.d. 

we get 



Eh' 




^ '^H',„,JmaxyyP,,, 


(71) 


which implies. 


^ [maxy^ Pi,, 


Hi,_,H',i,,] (72) 


Consequently, changing our feedback allocation hAf. will 
reduce the objective function of ( |42] i. 

iii) To complete the proof, lets consider two UTs (i, j) and 
(n, k) where n ^ i. We assume that according to our algorithm 
,- = 1 and = 0 (the other case of 5^,, = 0 and 

Kn k ~ ^ can also be deduced in the same way). If we change 
the allocation by setting b^k — 1 ™d &* i, = 0, the resulting 
objective function of will be smaller. This is due to the 
following; According to i) our algorithm selects the best UTs 
in cell i and according to ii) our algorithm selects the best UTs 
in other cells. Furthermore, in the second step of our algorithm, 
we compare between the selected UT in cell i and the selected 
one from other cells and select the UT that maximizes the 
objective function of (|42|). 


Part II : Proof of Corollary 

Recall the expression for Lyapunov drift from Proposition 
Recall that the expectation on the right hand side of (|2T| 
is taken across the random processes in the system (and in 
particular the randomness of the channel states). When the 
SCBS has only the estimate of the channel state, by the law 
of iterated expectations, we have 


Ay < Cl + ^ ^ + k'ijNo) + ^ ^ Eh Wi,jhi,i,,j 

(n,/c) 

= C + Qi,j{Xi,j + Vi.jNo) - E (73) 

where Ci is given as in Proposition [T] and the matrix 

i'lT'jk) k'n,kQn^kP^i^n^k based On 

Ai,3) 

the estimate of the CSI. For the feedback policy considered 
in this work, recall that if the UT„,fc feeds back its CSI to 
the SCBSi, then A,n,k = ^i,n,k if a UT„ ^ does not 
feed back its CSI to the SCBSi, then \ii.n,k = E[hi,„,fc]. 
Therefore, the matrix B^,, can be compactly written in terms 
of the feedback indicator as B^,, = (5i,, (6j,i,,Hi,i,, + 

^ (n,fc) k'ji^kQn,k{bi^ji^k^i,n,k F 'P^i,n,k) UI. 

Aid) 

As before, following the approach of Lyapunov optimiza¬ 
tion, we minimize the bound on the Lyapunov drift. Therefore, 
for a given CSI feedback strategy b, we consider the beam¬ 
forming vector to maximize the term max^ ,■ 
and then choose the feedback strategy to maximize the 
E[maxw and examine the performance of 
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our algorithm. Therefore we have, 


Ay < Cl + 

— maxE[max^^ 

= Cl + QijiKj + Vi jNpi) 
hi 

— maxE[maxj)] (74) 

i,i 

where ( |74l i follows from steps similar to that of Proposition 

2 . 

Using Theorem our feedback and allocation policy min¬ 
imizes the right hand side of and therefore minimizes 
the bound on the Lyapunov function. Replacing this by some 
other alternate feedback, beamforming and power allocation 
policy and we have. 


Ay < Cl + + VijNn) + UE[^^ PjL] 


E 


(75) 


i,3 {n,k) 


In particular choosing a stationary randomized policy such that 


E 


- E K3Qh3d3^ 3 + 


^,3 


(n,fc) 

^ih3) 


P Aij- + e 

n'£n3] = pd{e) 


(76) 

(77) 


we have 


Ay <Ci-e^Q,,,+UP.;f (78) 

From and following some straightforward steps (similar 
to Lemma 4.1, ll22ll . omitted here for brevity), we can conclude 
the result of Corollary 6. 
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